Introduction to GFS

The storage area network (SAN) system consists of a cluster of commodity storage devices connected to each other, providing block-level data storage to the clients over the network. SAN systems are easy to scale by adding more storage devices. However, these systems are difficult to manage because of the complexity of the second network — the Fiber Channel (FC). To set up the Fiber Channel, we need dedicated host bus adapters (HBAs) to be deployed on each host server, switches, and specialized cabling. It is difficult to monitor where failure has occurred in this complex system. Data inconsistency issues among replicas may appear in this case. Rebalancing the load on the storage devices might also be difficult to handle with this architecture.

Note: SAN deployments are special-purpose networks apart from the usual Ethernet networks. This duplicate network, while good for segregating storage traffic, is expensive in terms of dollar cost.

Functional requirements#

Functional requirements include the following:

• Data storage: The system should allow users to store their data on GFS.
• Data retrieval: The system should be able to give data back to users when they request it.

Non functional requirements#

Non functional requirements include the following:

• Scalability: The system should be able to store an increasing amount of data (hundreds of terabytes and beyond), and handle a large number of clients concurrently.
• Availability: A file system is one of the main building blocks of many large systems used for data storage and retrieval. The unavailability of such a system disrupts the operation of the systems that rely on it. Therefore, the file system should be able to respond to the client’s requests all the time.
• Fault tolerance: The system’s availability and data integrity shouldn’t be compromised by component failures that are common in large clusters consisting of commodity hardware.
• Durability: Once the system has acknowledged to the user that its data has been stored, the data shouldn’t be lost unless the user deletes the data themselves.
• Easy operational management: The system should easily be able to store multi-GB files and beyond. It should be easy for the system to handle data replication, re-replication, garbage collection, taking snapshots, and other system-wide management activities. If some data becomes stale, there should be an easy mechanism to detect and fix it. The system should allow multiple independent tenants to use GFS for safe data storage and retrieval.
• Performance optimization: The focus should be on high throughput rather than low latency for applications that require processing for large datasets. Additionally, Google’s applications, for which the system was being built, most often append data to the files instead of overwriting the existing data. So, the system should be optimized for append operations. For example, a logging application appends log files with new log entries. Instead of overwriting existing copies of the crawled data within a file, a web crawler appends new web crawl data to a crawl file. All MapReduce outputs write a file from beginning to end by appending key/value pairs to the file(s).

• Relaxed consistency model: GFS does not need to comply with POSIX standards because of the unique characteristics of the use cases/applications that it targets to serve. A file system must implement a strong consistency model in order to be POSIX compatible. In POSIX, random write is one of the fundamental operations. In GFS, there are more append operations and very few random writes. That’s why GFS doesn’t comply with POSIX and provides a relaxed consistency model that is optimized for append operations. Data consistency in a distributed setting is hard, and GFS carefully opts for a custom-consistency model for better scalability and performance.

Let’s explore GFS’s architecture, which enables it to fulfill the requirements mentioned above.

A GFS cluster consists of two major types of components– a manager node and a large number of chunkservers. It stores a file’s data in the form of chunks. The architecture is shown in the following illustration.

• Each file is split into fixed-size chunks. The manager assigns each chunk a 64-bit globally unique ID and assigns chunkservers where the chunk is stored and replicated. A manager is like an administrator that manages the file system metadata, including namespaces, file-to-chunk mapping, and chunk location. The metadata is stored in the manager’s memory for good performance. For a persistent record of the metadata, the manager logs the metadata changes in an operation log placed on the manager’s hard disk so that it can recover its state after the restart by replaying the operation log. Besides managing metadata, the manager also handles the following tasks:
  • Data replication and rebalancing
  • Operational locks to ensure data consistency
  • Garbage collection of the deleted data

  Note: Though Google calls this manager a “master” in their research paper on GFS, we will use the term “GFS manager”, “manager node”, or simply the “manager” to refer to the same thing.

Bird’s eye view#

In the coming lessons, we will design and evaluate GFS. The following concept map is a quick summary of the problem GFS solves and its novelties.